Modality Translation in Remote Sensing Time Series

Modality translation, which aims to translate images from a source modality target one, has attracted growing interest in the field of remote sensing recently. Compared translation problems multimedia applications, often suffers inherent ambiguities, i.e., single input image could correspond multiple possible outputs, and results may not be valid following interpretation tasks, such as classification change detection. To address these issues, we make attempt utilizing time-series data resolve ambiguities. We propose novel multimodality framework, exploits temporal information two aspects: 1) by introducing guidance given temporally neighboring modality, employ feature mask module transfer semantic output without requiring use any labels 2) while incorporating pairs time series, constraint is formulated during learning process order guarantee uniqueness prediction result. also build multimodal multitemporal dataset that contains synthetic aperture radar (SAR), visible, short-wave length infrared band (SWIR) series same scene encourage promote research on sensing. Experiments are conducted for cross-modality tasks (SAR visible SWIR). Both qualitative quantitative demonstrate effectiveness superiority proposed model.